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This is in response to the appeal brief filed 6/5/06 appealing from the Office action 



mailed 1 1/2/05. This supplemental Examiner's Answer is for including the Examiner's 



statement regarding the related appeals and interferences. 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

The amendment filed 2/2/06 was entered before appeal. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 

5,995,091 NEAR ET AL. 1 1 -1 999 
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5,943,046 



CAVE ET AL. 



8-1999 



5,892,847 



JOHNSON 



4-1999 



5,663,962 



CAIRE ET AL. 



9-1997 



Shaw et al., "Microsoft Office 6-in-1, Que Corporation 1994, pp. 379-380, 384-389, 396- 
402,419-425, 492-496 

(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

1. Claims 122-123. 125-129. 132. 134-136. 140, 143-146. 153-173 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over Near et al. (US Pat No. 5.995.091 . 
11/30/99. filed 5/10/96). 

Regarding independent claim 122, Near discloses: 

- receiving author specification of multimedia content and choreography 

information that indicates a temporal order for rendering the multimedia content, 
the multimedia content being defined by at least first, second, and third 
multimedia objects (figure 2, col 3, line 28 to col 4, line 15, col 9, lines 51-67: the 
arrangement of the interleaved playback includes media portions such as video 
chunk 1 , audio chunk 1 , video chunk 2, video chunk 3 where each portion has 
associated timestamp, engine destination, and message shows the 
choreography information associated with media portion and the temporal order 
of display among the objects; the authoring tool used by an author for specifying 
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images and sounds for playback at specified times implies the author 
specification of multimedia content and choreography information for rendering 
the multimedia content defined by at least first, second, and third multimedia 
objects) 

- interleaving data slides of the first and second multimedia objects with each other 
and adding the interleaved data slides to a multimedia document (col 9, lines 20- 
67) 

- adding the data slices of the first and second multimedia objects to the 
multimedia document without interleaving the data slices of the third multimedia 
object with data slices of other objects in the multimedia document (col 9, lines 
21-67, col 13, lines 1-34: interleaving multimedia objects and adding these 
objects to the multimedia document; some separate audio and video are 
combined to reproduce the playback output shows that said separate audio and 
video are added to the multimedia document playback without interleaving) 

- streaming the multimedia document to a recipient for rendering according to the 
choreography information (col 3, line 64 to col 4, line 61 , col 7, line 37 to col 8, 
line 20, col 12, line 52 to col 13, line 34) 

Near does not explicitly disclose said streaming is carried out such that the data slices 
of the first and second multimedia objects are progressively rendered before all data 
slices of the first and second multimedia objects are received and the data slices of the 
third multimedia object are progressively rendered only after a sufficient amount of the 
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data slices of the third multimedia object are received to enable rendering of the third 
multimedia object. 

However, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to have combined the feature of progressively rendering the first 
and second multimedia objects before all of their data slices are received, and 
progressively rendering the third multimedia object only after a sufficient amount of its 
data slices is received for the following reason. The interleaving of multimedia data was 
known to help speeding up the rendering process. Therefore, the first and second 
multimedia objects, which are interleaved, are rendered before all of their data are 
received. The third object, which is not interleaved, is rendered in a normal process, 
which means it is rendered when receiving all of its data. 

Regarding claim 123, which is dependent on claim 122, Near discloses that the data 
slices of the first, second, and third multimedia objects are rendered progressively 
together (col 3, line 28 to col 4, line 15, col 9, lines 51-67: playback the multimedia 
object shows that data of a plurality of multimedia objects including the first, second, 
and third objects are progressively rendered). 

Regarding claim 1 25, which is dependent on claim 1 22, Near discloses that all of the 
data slices of the third multimedia object are progressively rendered after any of the 
data slices of the first and second multimedia objects are progressively rendered (see 
claim 122). 
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Regarding claim 126, which are dependent on claim 122, Near discloses that the data 
slices of the first and second multimedia objects can be progressively rendered 
independent of bandwidth without degradation (col 2, lines 61-67: generating a 
bandwidth-controlled presentation data stream, thereby controlling network load and 
providing predictable performance on a variety of playback systems implies that the 
playback performance is not dependent on the bandwidth since the network load while 
sending media data is controlled via the authoring tool). 

Regarding claim 127, which is dependent on claim 122, Near discloses that the data 
slices of the third multimedia object cannot be progressively rendered independent of 
bandwidth without degradation (col 13, lines 1-34: since some specified audio and 
image files are separate from the other multimedia objects, so they are not interleaved 
with the other multimedia objects, rendering of these objects is performed when all of 
the data slices of these objects are received, and thus in a slower process). 

Regarding claim 128, which is dependent on claim 122, Near discloses that the data 
slices of the third multimedia object are rendered only after all of the data slices of the 
third multimedia object are received (as in claim 122, rendering of the third object 
occurs when all data slices of the third object are received). 
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Regarding claim 129, which is dependent on claim 122, Near discloses that the 
choreography information comprises an indication of the author's intent that the first, 
second and third files be rendered substantially together (col 3, line 32 to col 4, line 27). 

Regarding claims 132 and 134, which are dependent on claim 122, Near discloses that 
the first object comprises an image file (figure 2, the optional data in the video chunk 
can be image file). 

Regarding claim 135, which is dependent on claim 122, Near discloses that the third 
object comprises a sound file (col 3, line 56 to col 4, line 26). 

Regarding claim 136, which is dependent on claim 122, Near discloses that the third 
object comprises a video file (figure 2). 

Regarding claim 140, which is dependent on claim 122, Near discloses rendering the 
multimedia document in a window on a computer display at the recipient (col 13, lines 
23-34). 

Regarding claim 143, which is dependent on claim 122, Near discloses that each object 
has an address indicating a player that plays the object (col 8, lines 1-13; col 4, lines 47- 
61 identifying images and sounds to be reproduced during the playback implies 
identifying the address of the media element and the software that plays the media). 
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Regarding claim 144, which is dependent on claim 122, Near discloses compressing 
information in each object (col 8, lines 13-29, col 9, lines 20-39; col 4, lines 47-61). 

Regarding claim 145, which is dependent on claim 122, Near discloses creating an 
object in the file (col 9, lines 20-50) and locating player data within an object defining a 
player that plays the object (col 10, lines 40-67: playback media software used to 
identify individual portion of the playback data stream and the manager software calls a 
function for a player element of media data). 

Near does not disclose the created object is an unknown object. However, it would 
have been obvious to one of ordinary skill in the art at the time of the invention was 
made to have modified Near to include the created object being an unknown object for 
the following reason. Near defines the multimedia data for the playback data stream 
where each multimedia element is considered as an object in the data stream (figure 2, 
col 51-67). Thus no matter the object is known or unknown, the system always locates 
the player associated with the multimedia element. 

Regarding claim 146, which is dependent on claim 122, Near discloses two or more of 
the objects have at least one common attribute, including at least one of a command for 
perception of the objects, an ability to pass and receive message, and an ability to 
supply and retrieve the data embodied in the objects (col 4, lines 47-61: the fact that 
each multimedia object is identified by the interleaver element implies that the 
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multimedia objects have a command for perception the object, which is a common 
attribute of the objects). 

Regarding claim 153, which is dependent on claim 122, Near discloses that the 
temporal order is independent of a recipient input (col 3, line 64 to col 4, line 27, col 7, 
lines 50-64: the fact that the playback of the images and sound data stream with 
specified times is generated by an author for a selected bandwidth shows that the 
temporal order is defined by the author, and thus is maintained independent of a 
recipient). 

Regarding claim 154, which is dependent on claim 122, Near discloses that the 
temporal order is independent of a recipient hardware configuration (col 3, lines 48-55: 
since the temporal order is defined by an author, the author controls the playback, and 
thus independent of the recipient hardware configuration). 

Regarding claim 155, which is dependent on claim 122, Near discloses that the 
temporal order is independent of a recipient software configuration (col 3, lines 48-55: 
since the temporal order is defined by an author, the author controls the playback, and 
thus independent of the recipient software configuration). 

Regarding claim 156, which is dependent on claim 122, Near discloses that the media 
content is rendered independent of an author-specified bandwidth to be used to send 
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the multimedia document (col 8, lines 1-28: no matter the bandwidth selected by the 
author used to send the multimedia document is sufficient or insufficient, rendering of 
the multimedia document is still performed via the playback). 

Claims 157-164 are for a computer device to perform the method claim 122-128, 137, 
and are rejected under the same rationale. 

Claim 165 is for a computer device to perform the method claim 156, and is rejected 
under the same rationale. 

Claims 166-172 are for the document included in method claim 122-128, and are 
rejected under the same rationale. 

Claim 173 is for the document included in method claim 156, and is rejected under the 
same rationale. 

2. Claims 131 and 133 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Near as applied to claim 122 above, and further in view of Cave et al. (US Pat No. 
5.943.046. 8/24/99, filed 7/19/95). 



Regarding claims 131 and 133, which are dependent on claim 122, Near discloses a 
multimedia system for performance playback multimedia data (col 3, lines 28-47). Near 
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does not disclose explicitly that the first and second multimedia objects comprises a text 
file. 

Cave discloses that the multimedia objects include voice, music, pictures and text (col 

2, lines 60-67). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Cave into Near for incorporate text files into the 
multimedia objects for having different types of objects in the multimedia documents. 

3. Claims 141. 147-148 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Near as applied to claim 140 above, and further in view of Shaw et al., Microsoft 
Office 6-in-L Que Corporation 1994, pages 379-380, 384-389, 396-402, 419-425, 492- 
496. 



Regarding claim 141 , which is dependent on claim 140, Near does not disclose: 

- creating an exclusionary area within the window 

- locating an object within the exclusionary area, the object being selected from a 
group of objects including a framed image, a slide show, framed text, sound data, 
a separator, or a hyperlink 

Shaw discloses: 

- creating an exclusionary area within the window (page 401 , figure 4.4) 

- locating an object within the exclusionary area, the object being selected from a group 
of objects including a framed image, a slide show, framed text, sound data, a separator, 
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or a hyperlink (page 401 , figure 4.4: the data in the area within the window can be text 
and graphics). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Shaw into Near since Shaw teaches the window features 
for locating objects selected from a group of specific objects providing the advantage to 
include in Near to enhance the use of objects from internet instead of merely video or 
audio from the media database. 

Regarding claims 147 and 148, which are dependent on claims 122 and 147 

respectively, Near does not disclose the document forms a code segment that receives 

image information, and wherein the image information is used to construct an image 

frame for a framed image that is part of the multimedia document. 

Shaw discloses that the document forms a code segment that receives image 

information, and wherein the image information is used to construct an image frame for 

a framed image that is part of the multimedia document (pages 400-401). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 

was made to have combined Shaw into Near since Shaw a code segment to control the 

receiving of image information as well as the structure of the media stream providing the 

advantage of utilizing such control to edit the media stream in Near as desired. 
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4. Claim 142 is rejected under 35 U.S.C. 103(a) as being unpatentable over Near 
as applied to claim 122 above, and further in view of Johnson (US Pat No. 5,892,847, 
4/6/99. filed 4/22/96). 

Regarding claim 142, which are dependent on claim 122, Near does not disclose 
defining as well as locating the update splash image within the data file. 
Johnson discloses: 

- splash image data defining a splash image and locating the splash image data within 
the data file for displaying the splash image on the computer display (col 4, lines 30-50) 

- further updating the splash image to be displayed (col 4, lines 30-63) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Johnson into Near since Johnson teaches the process of 
displaying of a splash image, which is an element of a multimedia document providing 
the advantage of including a splash image into the image file to enhance the image 
characteristics in Near. 

5. Claims 149-152 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Near as applied to claim 122 above, and further in view of Caire et al. (US Pat No. 
5.663,962. 9/2/97. filed 9/15/95). 



Regarding claim 149, which is dependent on claim 122, Near discloses a header, 
timestamps, and associated data for a data stream (col 5, line 64 to col 6, line 16). 
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Near does not disclose that the choreography information further comprises an object 
archive for storing information about one or more objects, the object archive including 
information about the relationship of the object file with the document, and a multiplex 
section including data for the objects in the document. 
Caire discloses: 

- a header (col 1 , lines 65 to col 2, lines 1-2, each packet in the overall stream 
includes a header) 

- an object archive for storing information about the plurality of object files, the 
object archive including information about the level of each object file with the 
hierarchy (col 1, lines 65 to col 2, lines 1-2, each packet of the multimedia stream 
stores information; col 1 , lines 37-52, it is desired for instance to insert into the 
complete stream also some subtitles to be displayed during the presentation....) 

- a multiplex section including data for each of the object files of the document (col 
1, lines 65 to col 2, lines 1-9, 45-59) 

- the object files in the multiplex section are each played by a player as the 
multiplex object file is received by a receiver (col 1 , lines 65 to col 2, lines 1-2) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Near since Caire provides the choreography 
and the multiplexing features for a multimedia presentation. The combination of Caire 
into Near would provide more detailed options in the relationship of the objects in a 
multimedia document to effectively controlling and changing the presentation of the 
objects. 
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Regarding claims 150 and 151, which are dependent on claims 149 and 150 
respectively, Near does not disclose an object number counter indicating the number of 
objects, a plurality of object descriptions, each object description describing a 
corresponding one of the objects, and a choreography group providing information 
about a first group of objects, a group object counter indicating the number objects in 
the choreography group, size and type data for each object, header data, data slices of 
the objects interleaved together, and placing one or more slice size data blocks before 
one or more of the interleaved data slices, each slice size data block corresponding to a 
data slice and providing a size of the corresponding data slice. 
Caire discloses: 

- an object number counter indicating the number of object files (col 2, lines 10-20) 

- a plurality of object descriptions, each object description describing a 
corresponding one of the object files (col 1 , lines 65 to col 2, lines 1-2, the 
header includes information of the type of a packet in the multimedia stream) 

- a choreography group providing information about a first group of object files (col 
1 , lines 65 to col 2, lines 1-2, packets of different types are included in the overall 
stream as a sequence of intervals wherein the type of a packet is disclosed in the 
heading are considered as a choreography group providing information about the 
object files) 

- size and type data for each object file (col 1 , lines 65 to col 2, lines 1-2, data type 
of each packet in the multimedia stream) 

- header data (col 1 , lines 65 to col 2, lines 1-2, each packet includes a header) 
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- the data slices of the object files interleaved together (col 1 , lines 65 to col 2, 
lines 1-2, the overall stream is structured as a sequence of intervals called 
packets, each of which contains data of single type, indicated in a header of the 
packet itself; since data of different types are arranged in the sequence of 
intervals called packets, the packets which are equivalent to the object files, are 
interleaved together) 

- a first player pointer including an address of a player that plays the choreography 
group (col 2, lines 3-9, for each interval, the multiplexer has to decide from which 
the input stream it should take the data in order to construct the packets; this 
implies that the multiplexer has to decide where to point to play the overall 
stream which is equivalent to the choreography group as mentioned above) 

- locating a plurality of slice size data blocks before the interleaved data slices, 
each slice size data block corresponding to one of the data slices and providing a 
size of the corresponding data slice (col 4, lines 45-53, the number of data bytes 
and the number of header bytes in each packet show the size of each packet 
which is equivalent to the data block) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Near since Caire provides the choreography 
and the multiplexing features for a multimedia presentation. The combination of Caire 
and Near would provide the relationship of the objects in a multimedia document for 
effectively controlling and changing the presentation of the objects. 
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Regarding claim 152, which is dependent on claim 150, Near does not disclose a non- 
multiplex section following the multiplex section where the non-multiplex section 
includes one or more separate objects that are not played by a player as the separate 
object files are received by a receiver. 

Caire discloses a plurality of separate object files that are not played by a player as the 
separate object files are received by a receiver (col 1 , lines 37-45, ...video and audio 
information have to be separated again, by an inverse of demultiplexing process, as 
presentation occurs on different devices...). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Near since Caire provides the choreography 
and the multiplexing features for a multimedia presentation. The combination of Caire 
and Near would provide the relationship of the objects in a multimedia document for 
effectively controlling and changing the presentation of the objects. 

(10) Response to Argument 

Appellant argument regarding the 112, first paragraph rejections of claims 124 and 130 
are persuasive. The 112, first paragraph rejections of claims 124 and 130, thus, are 
withdrawn. 

Appellant argue that Near only describes playback of interleaved image and/or audio 
data, not playback of non-interleaved image and/or audio data. In Near reference, there 
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is no teaching of adding multimedia data objects "without interleaving" to the multimedia 
document. 

Examiner respectfully disagrees. 

Though Near discloses interleaving multimedia data (col 9, lines 20-67), Near also 
discloses reproducing multimedia that does not require interleaving multimedia since 
interleaving is not mentioned (col 13, lines 12-34). 

In Near, the separate track audio playback are combined by the audio mixer to form the 
combined output for reproducing multimedia output, several video track playback 
outputs are combined for reproducing multimedia output where the combination of audio 
tracks and the combination of video tracks for reproducing multimedia data do not 
require interleaving of video and audio data (col 13, lines 12-34). These combinations, 
therefore, are considered adding multimedia data to a multimedia document without 
interleaving multimedia data. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the 
Related Appeals and Interferences section of this examiner's answer. 
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For the above reasons, it' is believed that the rejections should be sustained. 



Respectfully submitted, 



Cong-Lac Huynh 
Primary Examiner 
Art Unit 2178 
7/26/06 



Conferees 





Stephen H 

STEP^EN^ 0 * 16 
SUPERVISORY PATENT EXAMINER 



